Scientists are on the verge of opening the Large Hadron Collider (LHC), which will use ultra-powerful magnets to race proton beams around a 17-mile circular underground tunnel and smash them into each other 40-million times a second. These collisions will produce tiny particles not seen since just after the Big Bang and perhaps will enable scientists to find the elusive Higgs boson, which – if theories are correct – endows all objects with mass. The LHC may also help scientists figure out why all the matter in the universe wasn’t destroyed by anti-matter, which would have been inconvenient for those who enjoy residing in a universe that isn’t a great vacuum devoid of life.
The WLCG was set up to distribute the mountains of data produced by the seemingly infinite number of particle collisions. Data will be gathered from CERN and distributed to thousands of scientists throughout the world.
One writer described the grid as a “parallel Internet.” Ruth Pordes, executive director of the Open Science Grid (OSG), which oversees the US infrastructure for the LHC network, describes it as an “evolution of the Internet.” New fiber-optic cables with special protocols will be used to move data from CERN to 11 Tier1 sites around the globe, which in turn use standard Internet technologies to transfer the data to more than 150 Tier2 centers.
The data is first produced in the collisions, which occur in caverns 100 meters underground. When proton beams collide and produce new particles, data will be read from 150-million sensors and sent to a counting room where signals are filtered to produce “raw data.”
Raw data will be sent over dedicated 10Gbps networks to the CERN Computer Center, known as Tier0. This data will be sent to tape storage and also to a CPU farm, which will process information and generate “event summary data.” Subsets of both the raw data and summaries will then be sent to the 11 Tier1 sites, including Brookhaven National Laboratory on Long Island, NY, and Fermilab in Illinois.
Each of the 11 Tier1 centers connects to CERN via a dedicated 10Gbps link. A general-purpose research network connects the Tier1 centers to each other. Each Tier1 center receives only certain subsets of information. Brookhaven, for example, is dedicated to ATLAS, one of several large detectors housed at the LHC, while Fermilab handles data from the CMS (Compact Muon Solenoid) detector.
Tier1 centers are responsible for reprocessing raw data, which is then kept on local disk and tape storage and distributed to Tier2 centers, located in most parts of the world. General-purpose research networks, such as the US ESnet, connect Tier2 centers to Tier1 sites, and to each other. Tier2s are located mainly at universities, where physicists will analyze LHC data. Ultimately, about 7,000 physicists will scrutinize LHC data for information about the origins and makeup of our Universe.
The LHC collisions will produce 10- to 15-Petabytes of data a year, says Michael Ernst of Brookhaven National Laboratory, where he directs of the program that will distribute data from the ATLAS detector.
Worldwide, the LHC computing grid will be comprised of about 20,000 servers, primarily running the Linux operating system. Scientists at Tier2 sites can access these servers remotely when running complex experiments based on LHC data. If scientists need a million CPU hours to run an experiment overnight, the distributed nature of the grid allows them to access that computing power from any part of the worldwide network. With the help of Tier1 sites, the goal is to make using the grid just as easy for universities as using their own internal networks.
Compact Muon Solenoid (CMS) Readies Network Links for LHC Data
Transfer 300 thousand million bytes (300 Gigabytes) per day for 6 out of 7 consecutive days and move a total of 2.3 Terabytes during that same 7-day period – that’s 2.3 million million bytes – these are the criteria that each major network link in the CMS’s computing structure must satisfy when the LHC turns on. Together they will transfer tens of Terabytes a day. The CMS computing structure comprises an internationally distributed system of services and resources that interact over the network through the WLCG.
In July 2007, CMS assembled a task force to ready the inter- and intra-tier networks. The task force ran a set of simulated datasets up the computing chain to CERN, and ran workflows based on these datasets back down to the Tier1 and Tier2 centers. Thanks to a tool developed by Brian Bockelman of the University of Nebraska and CERN summer student Sander Sõnajalg, the team extracted transfer volume data from PheDEX, CMS’s data transfer middleware, and applied the stringent commissioning criteria to it. The transfer tests in February 2008 more than doubled the throughput over all links seen in the 2007 CSA07 tests.” said Letts. As of May 2008, all the Tier1 sites are stable and about two-thirds of the Tier1 to Tier2 connections have been commissioned.
Below is an illustration of Tier1 (red) and Tier2 (blue) sites worldwide in CMS. (Image courtesy of James Lett
www.isgtw.org/images/CMSFig2L.jpg)
ALICE Computing
In addition to accelerating and colliding protons, the LHC will collide high-energy lead ions one month each year primarily for ALICE (A Large Ion Collider Experiment). ALICE scientists hope to use these collisions to recreate inside the detector tiny drops of primordial matter, the so-called quark-gluon plasma, that presumably existed a few microseconds after the Big Bang.
Before the universe cooled enough for quarks to be bound inside protons and neutrons as they are today, quarks (elementary particles that constitute matter together with electrons) and gluons (particles that carry the strong force that holds nuclear matter together) moved around freely in a kind of plasma state. ALICE collaborators will study this primordial plasma to gain a better understanding of the early universe and the strong force.
ALICE is expected to produce around 100 Terabytes of data each day – the equivalent of about 20,000 DVDs. Processing this data will require extensive computing power – roughly 10,000 CPUs running continuously. The WLCGwill enable the raw detector data to be distributed among 1,000 ALICE collaborators in 31 countries.In preparation for the outpouring of ALICE data, collaborators have been writing data analysis software and purchasing the necessary computers.
The US will handle just under 10% of ALICE’s computing, and hardware outlays are currently at four US institutions: the NERSC computing facility (affiliated with Lawrence Berkeley National Laboratory), Lawrence Livermore National Laboratory, the Ohio State Supercomputing Center, and the Texas Learning and Computation Center (affiliated with the University of Houston).
URL:
www.networkworld.com/news/2008/042208-large-hadron-collider.html?page=1
www.isgtw.org/?pid=1001047
http://lcg.web.cern.ch/LCG/
www.isgtw.org/?pid=1001103
www.isgtw.org/?pid=1001560
